Skip to content
This repository has been archived by the owner on Jul 12, 2023. It is now read-only.

JSON CEE structured logging #17

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

dpocock
Copy link

@dpocock dpocock commented Jul 4, 2021

What is the current behavior you want to change?

Add support for structured logging in the JSON CEE format.
This makes it much easier to load the logs into tools like OpenSearch.
This makes it much easier to combine logs from multiple application servers (media, SIP, XMPP, TURN, web servers) and merge the logs from all processes with events in time order.
I contributed similar patches for reSIProcate and GStreamer already, it is well documented in the GStreamer pull request:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/847
resiprocate/resiprocate@d1df9ae

What is the new behavior provided by this change?

When the environment variable GST_DEBUG_STRUCTURED=JSON_CEE exists, each log line will be in JSON format, following the CEE conventions, like this:

{
    "appname": "foo",
    "file!line": 519,
    "file!name": "gstregistrychunks.c",
    "hostname": "lab1.example.org",
    "msg": "Added pad_template sink",
    "native!function": "gst_registry_chunks_load_pad_template",
    "native!object": "",
    "native!time!elapsed": 679585795,
    "pname": "/usr/bin/kurento-media-server",
    "pri": "DEBUG",
    "syslog!level":6,
    "gstreamer!level":5,
    "proc!id": "9963",
    "proc!tid": 93905064053760,
    "subsys": "GST_REGISTRY",
    "time": "2021-07-04T12:58:21.076937000Z"
}

How has this been tested?

Tested with Kurento as daemon, validated the log files with python -mjson.tool
Tested with Kurento in foreground, validated the output with python -mjson.tool

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature / enhancement (non-breaking change which improves the project)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • My change requires a change to the documentation
  • My change requires a change in other repository

When running as a daemon, the Boost log formatting function in main/logging.cpp is used. This doesn't depend on any change to GStreamer.

When running in the foreground, Kurento relies on the code in GStreamer to write and format the log lines. Therefore, I backported the change from GStreamer upstream for Kurento's GStreamer fork, I'll submit that pull request too.

Checklist

  • I have read the Contribution Guidelines
  • I have added an explanation of what the changes do and why they should be included
  • I have written new tests for the changes, as applicable, and have successfully run them locally

@dpocock dpocock changed the title Pocock/json cee JSON CEE structured logging Jul 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant